Contextual-based Image Inpainting: Infer, Match, and Translate

We study the task of image inpainting, which is to fill in the missing region of an incomplete image with plausible contents. To this end, we propose a learning-based approach to generate visually coherent completion given a high-resolution image with missing components. In order to overcome the difficulty to directly learn the distribution of high-dimensional image data, we divide the task into inference and translation as two separate steps and model each step with a deep neural network. We also use simple heuristics to guide the propagation of local textures from the boundary to the hole. We show that, by using such techniques, inpainting reduces to the problem of learning two image-feature translation functions in much smaller space and hence easier to train. We evaluate our method on several public datasets and show that we generate results of better visual quality than previous state-of-the-art methods.Comment: ECCV 2018 camera read

Dynamical Modulation of Wintertime Synoptic-Scale Cyclone Activity over the Japan Sea due to Changbai Mountain in the Korean Peninsula

The dynamical impact of the Changbai Mountain Range in the Korean Peninsula on the extratropical cyclone activity over the Japan Sea in early winter is examined using the Weather Research Forecasting model. We have conducted two independent long-term integrations over 15 winter months (December only) from 2000 to 2014 with and without modified topography. The results show that the Changbai Mountain Range plays a vital role in increasing cyclone track frequency, low-level poleward eddy heat flux, and the local deepening rate over the Japan Sea through enhancement of the lower-tropospheric baroclinic zone (LTBZ). This mountain range gives rise to activation of the synoptic-scale cyclone activity over that region. From our case study on three typical cyclones, it is found that mesoscale structures in the vicinity of a cyclone’s center are dynamically modulated when it passes through the LTBZ and that cyclogenesis is triggered around that zone. A vorticity budget analysis shows that the stretching term relevant to enhanced low-level convergence plays a dominant role in intensifying cyclonic vorticities. We confirmed that the composite features of the three typical cases are consistent with the statistical ones of the dynamical modulation of the Changbai Mountain on synoptic-scale cyclone activity

Controllable Multi-domain Semantic Artwork Synthesis

We present a novel framework for multi-domain synthesis of artwork from semantic layouts. One of the main limitations of this challenging task is the lack of publicly available segmentation datasets for art synthesis. To address this problem, we propose a dataset, which we call ArtSem, that contains 40,000 images of artwork from 4 different domains with their corresponding semantic label maps. We generate the dataset by first extracting semantic maps from landscape photography and then propose a conditional Generative Adversarial Network (GAN)-based approach to generate high-quality artwork from the semantic maps without necessitating paired training data. Furthermore, we propose an artwork synthesis model that uses domain-dependent variational encoders for high-quality multi-domain synthesis. The model is improved and complemented with a simple but effective normalization method, based on normalizing both the semantic and style jointly, which we call Spatially STyle-Adaptive Normalization (SSTAN). In contrast to previous methods that only take semantic layout as input, our model is able to learn a joint representation of both style and semantic information, which leads to better generation quality for synthesizing artistic images. Results indicate that our model learns to separate the domains in the latent space, and thus, by identifying the hyperplanes that separate the different domains, we can also perform fine-grained control of the synthesized artwork. By combining our proposed dataset and approach, we are able to generate user-controllable artwork that is of higher quality than existingComment: 15 pages, accepted by CVMJ, to appea

Adaptive occlusion sensitivity analysis for visually explaining video recognition networks

This paper proposes a method for visually explaining the decision-making process of video recognition networks with a temporal extension of occlusion sensitivity analysis, called Adaptive Occlusion Sensitivity Analysis (AOSA). The key idea here is to occlude a specific volume of data by a 3D mask in an input 3D temporal-spatial data space and then measure the change degree in the output score. The occluded volume data that produces a larger change degree is regarded as a more critical element for classification. However, while the occlusion sensitivity analysis is commonly used to analyze single image classification, applying this idea to video classification is not so straightforward as a simple fixed cuboid cannot deal with complicated motions. To solve this issue, we adaptively set the shape of a 3D occlusion mask while referring to motions. Our flexible mask adaptation is performed by considering the temporal continuity and spatial co-occurrence of the optical flows extracted from the input video data. We further propose a novel method to reduce the computational cost of the proposed method with the first-order approximation of the output score with respect to an input video. We demonstrate the effectiveness of our method through various and extensive comparisons with the conventional methods in terms of the deletion/insertion metric and the pointing metric on the UCF101 dataset and the Kinetics-400 and 700 datasets.Comment: 11 page

Autonomous Navigation, Guidance and Control of Small Electric Helicopter

In this study, we design an autonomous navigation, guidance and control system for a small electric helicopter. Only small, light-weight, and inaccurate sensors can be used for the control of small helicopters because of the payload limitation. To overcome the problem of inaccurate sensors, a composite navigation system is designed. The designed navigation system enables us to precisely obtain the position and velocity of the helicopter. A guidance and control system is designed for stabilizing the helicopter at an arbitrary point in three-dimensional space. In particular, a novel and simple guidance system is designed using the combination of optimal control theory and quaternion kinematics. The designs of the study are validated experimentally, and the experimental results verify the efficiency of our navigation, guidance and control system for a small electric helicopter.ArticleINTERNATIONAL JOURNAL OF ADVANCED ROBOTIC SYSTEMS. 10:54 (2013)journal articl